System and method for local three dimensional volume reconstruction using a standard fluoroscope

ABSTRACT

A system for constructing fluoroscopic-based three-dimensional volumetric data of a target area within a patient from two-dimensional fluoroscopic images including a structure of markers, a fluoroscopic imaging device configured to acquire a sequence of images of the target area and of the structure of markers, and a computing device. The computing device is configured to estimate a pose of the fluoroscopic imaging device for at least a plurality of images of the sequence of images based on detection of a possible and most probable projection of the structure of markers as a whole on each image of the plurality of images. The computing device is further configured to construct fluoroscopic-based three-dimensional volumetric data of the target area based on the estimated poses of the fluoroscopic imaging device.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation-in-part of U.S. application Ser. No.15/224,812, filed on Aug. 1, 2016 which claims priority to U.S.Provisional Application Ser. No. 62/201,750, filed on Aug. 6, 2015. Thedisclosure of each of the prior applications is considered part of andis incorporated by reference in the disclosure of this application.

BACKGROUND Technical Field

The present disclosure relates to a system, apparatus, and method ofnavigation, position confirmation and position correction for surgicalprocedures. More particularly, the present disclosure relates to asystem and method for constructing a fluoroscopic-basedthree-dimensional volume from two dimensional fluoroscopic imagescaptured using a standard fluoroscopic imaging device and for utilizingthis fluoroscopic-based three dimensional volume for real-timedetermination of the location of a medical device relative to a targetwithin a patient.

Description of Related Art

There are several commonly applied medical methods, such as endoscopicprocedures or minimally invasive procedures, for treating variousmaladies affecting organs including the liver, brain, heart, lung, gallbladder, kidney and bones. Often, one or more imaging modalities, suchas magnetic resonance imaging, ultrasound imaging, computed tomography(CT), as well as others are employed by clinicians to identify areas ofinterest within a patient and ultimately targets for treatment.

For example, an endoscopic approach has proven useful in navigating toareas of interest within a patient, and particularly so for areas withinluminal networks of the body such as the lungs. To enable theendoscopic, and more particularly the bronchoscopic, approach in thelungs, endobronchial navigation systems have been developed that usepreviously acquired MRI data or CT image data to generate a threedimensional rendering or volume of the particular body part such as thelungs. In particular, previously acquired images, acquired from an MRIscan or CT scan of the patient, are utilized to generate a threedimensional or volumetric rendering of the patient.

The resulting volume generated from the MRI scan or CT scan is thenutilized to create a navigation plan to facilitate the advancement(e.g., manually or via a console, when robotic surgery is performed) ofa navigation catheter (or other suitable device) through a bronchoscopeand a branch of the bronchus of a patient to an area of interest.Electromagnetic tracking may be utilized in conjunction with the CT datato facilitate guidance of the navigation catheter through the branch ofthe bronchus to the area of interest. In certain instances, thenavigation catheter may be positioned within one of the airways of thebranched luminal networks adjacent to, or within, the area of interestto provide access for one or more medical instruments.

As another example, minimally invasive procedures, such as laparoscopyprocedures, including robotic-assisted surgery, may employintraoperative fluoroscopy in order to increase visualization, e.g., forguidance and lesion locating, or in order to prevents injury andcomplications.

In order to generate a navigation plan, or in order to even generate athree dimensional or volumetric rendering of the patient's anatomy, suchas the lung, a clinician is required to utilize an MRI system or CTsystem to acquire the necessary image data for construction of the threedimensional volume. An MRI system or CT-based imaging system isextremely costly, and in many cases not available in the same locationas the location where a navigation plan is generated or where anavigation procedure is carried out.

A fluoroscopic imaging device is commonly located in the operating roomduring navigation procedures. The standard fluoroscopic imaging devicemay be used by a clinician to visualize and confirm the placement of atool or a medical device after it has been navigated to a desiredlocation. However, although standard fluoroscopic images display highlydense objects such as metal tools and bones as well as large soft-tissueobjects such as the heart, the fluoroscopic images have difficultyresolving small soft-tissue objects of interest such as lesions.Further, the fluoroscope image is only a two dimensional projection. Inorder to be able to see small soft-tissue objects in three dimensionalspace, an X-ray volumetric reconstruction is needed. Several solutionsexist that provide three dimensional volume reconstruction ofsoft-tissues such as CT and Cone-beam CT which are extensively used inthe medical world. These machines algorithmically combine multiple X-rayprojections from known, calibrated X-ray source positions into threedimensional volume in which the soft-tissues are visible.

In order to navigate tools to a remote soft-tissue target for biopsy ortreatment, both the tool and the target should be visible in some sortof a three dimensional guidance system. The majority of these systemsuse some X-ray device to see through the body. For example, a CT machinecan be used with iterative scans during procedure to provide guidancethrough the body until the tools reach the target. This is a tediousprocedure as it requires several full CT scans, a dedicated CT room andblind navigation between scans. In addition, each scan requires thestaff to leave the room. Another option is a Cone-beam CT machine whichis available in some operation rooms and is somewhat easier to operate,but is expensive and like the CT only provides blind navigation betweenscans, requires multiple iterations for navigation and requires thestaff to leave the room.

Accordingly, there is a need for a system that can achieve the benefitsof the CT and Cone-beam CT three dimensional image guidance without theunderlying costs, preparation requirements, and radiation side effectsassociated with these systems. Furthermore, there is a need for a systemand a method which facilitate relatively accurate navigation of medicaldevices in relation to a target and in particular to facilitate aneffective treatment of the target.

SUMMARY

The present disclosure is directed to a system and method forconstructing three dimensional volumetric data in which smallsoft-tissue objects are visible from a video stream (or plurality ofimages) captured by a standard fluoroscopic imaging device available inmost procedure rooms. The fluoroscopic-based constructed threedimensional volumetric data may be used for guidance, navigationplanning, improved navigation accuracy, navigation confirmation, andtreatment confirmation.

Soft tissue objects are not visible in standard fluoroscopic images andvideo because they are obscured by dense objects such as dense tissue.The present disclosure is directed to a system and method for creating athree-dimensional pseudo-volume in which the obscuring objects arefiltered based on the three dimensional position, and then projectedback into two dimensions. In one aspect, multiple fluoroscopic images,each captured at a different angle, are utilized to construct the threedimensional pseudo-volume data. The present disclosure describes asystem and method which is capable of constructing the three dimensionalpseudo-volume data utilizing fluoroscopic images captured from a shortrange of angles relative to a patient or target region of a patient.Additionally, the present disclosure is also directed to a system andmethod for improving a previously created three dimensional renderingutilizing two dimensional images, or video, captured by a standardfluoroscopic imaging device.

As described in greater detail below, one aspect of the presentdisclosure is to determine three dimensional positions of features inthe fluoroscopic video, such as three dimensional catheter position,three dimensional target tissue (for example a lesion) position, etc. Inorder to accomplish this, the pose of the fluoroscopic imaging devicefor each frame must be determined or known. If an external anglemeasurement device coupled to the fluoroscopic imaging device isutilized, then the angles and pose of the imaging device is known foreach frame. However, when external measurement devices are not utilizedother techniques are employed to determine the poses of the fluoroscopicimaging device for each frame, as described in greater detail below. Forexample, previously acquired CT scan data may be registered to thefluoroscopic video in order to algorithmically find the pose of thefluoroscope for each frame of the captured video. Alternatively, bytracking a few visible markers (two dimensional visible features) in thefluoroscopic video, the pose and three dimensional positions may besolved together by using some structure from motion technique. In someaspects, a structure of markers placed externally to the patient may beutilized. The poses of the fluoroscope may be then estimated based ondetection of a possible and most probable projection of the structure ofmarkers as a whole on each image of the plurality of images In someaspects, easier techniques in which the poses are known (by an anglemeasurement device) may be utilized and only the three dimensionalfeatures' positions need to be solved. In order to correct formovements, at least one marker (or a surgical device such as a cathetertip) may be utilized.

Multiple fluoroscope two dimensional images can be processedalgorithmically to create pseudo three dimensional volumetric data,similar to Cone-beam CT, but with varying arbitrary angle range andfluoroscope poses, where the fluoroscope is rotated manually. The anglerange may be very small (˜30°) which may result in poor threedimensional reconstruction quality. The algorithm used is an iterativeaccelerated projection/back-projection method which unlike the analyticalgorithms (e.g., Radon transform, FDK) doesn't assume any predeterminedangle range or angle rotation rate. In order to overcome the poor threedimensional reconstruction, instead of displaying the raw threedimensional reconstructed data to the user, the three dimensionalreconstruction may be cropped around the area of interest (also referredto herein as the “FluoroCT Blob”). The cropped three dimensional datamay be then reprojected into two dimensional virtual fluoroscope imagesin which local soft-tissue features are visible, in particular. Thereprojected two dimensional images are of good quality (compared to thepoor three dimensional reconstruction), especially if the projectionsare done from the same fluoroscope poses as were seen in the video.

In order to reconstruct the three dimensional data, the pose of thefluoroscopic imaging device must be determined for each two dimensionalfluoroscope frame in the video, relative to some fixed coordinatesystem. The pose of the fluoroscopic imaging device for each framecaptured may be determined using any of the methods described below. Forexample the pose may be determined using an external measurement devicecoupled to the fluoroscopic imaging device or by utilizing a structureof markers places externally to the patient and with respect to thetarget.

Additionally, or alternatively, the pose may be determined using anexternal measurement device and a single marker or a catheter tip.Specifically, in some cases, the fluoroscope (or C-arm) may jitterduring rotation, in which case some stabilization is needed.Unfortunately, due to filtering, the angle measurement device may stillreport smooth angles throughout the video, ignoring the high-frequencieswhich are present in the actual camera poses. In this case, a single twodimensional marker can be tracked throughout the video and used tostabilize the camera poses, or to increase their accuracy at the area ofthe marker. Since the marker will usually be located at the region ofinterest, this increases the camera pose accuracy in this region, andthus improves the three dimensional reconstruction quality. This methodcan also be used to compensate for patient body movement such asbreathing during the video. Instead of using the camera pose as reportedby the angle measurement device, a compensated camera pose is computedusing the tracked two dimensional marker, such that all camera poseswill be correct relative to it. The single marker used can be the tip ofa tool or a catheter which is currently inserted to the patient.

Additionally, or alternatively, the pose may be determined viaregistration of the fluoroscopic video to previously acquired CT data.Specifically, a previously acquired CT of the patient may be available.In this case, each frame of the fluoroscope video can be registered to avirtual fluoroscopic frame of the CT (camera pose is searched in CTspace until the virtual fluoroscopic image, corresponding to the camerapose, matches the one seen in the video). In this way, the camera poseis realized using image-based features matching.

Additionally, or alternatively, the pose may be determined via astructure of markers. Specifically, the structure of markers ispositioned externally to the patient, e.g., under the patient, whilecapturing the images. The structure of markers is further positionedsuch that each image includes a projection of at least a portion of thestructure. A probability map may be then generated for each imageindicating the probability of each pixel in the image to be a marker ofthe structure of markers. Multiple virtual candidate for the projectionof the structure of markers on the image may be generated by virtuallypositioning the fluoroscope in possible different locations, includingpossible orientations. The candidate having the highest probability ofbeing the projection of the structure of markers on the image may bethen identified based on the probability map. The virtual pose of thefluoroscope associated with the identified candidate may be thendetermined as the estimated pose of the fluoroscope while capturing theimage. Optionally, the process of identifying a candidate may berefined. A locally deformed version of the candidate may be generatedbased on the probability map in order to maximize its probability ofbeing the projection of the structure of markers on the image. A newvirtual candidate may be then fitted to the locally deformed version ofthe identified candidate. The virtual pose of the fluoroscope whichwould generate the new improved candidate is the calculated anddetermined as the estimated pose of the fluoroscope while capturing theimage.

Additionally, or alternatively, the camera pose may be determined usingstructure from motion techniques. Specifically, if numerous twodimensional features can be tracked throughout the two dimensional videoframes, from beginning to end, then these two dimensional features canbe used to realize camera poses (together with three dimensional featurepositions) for each frame. These features can be artificial markerswhich were introduced to the patient during procedure.

Filtering the obscuring tissue from the tissue of interest can be doneby cropping at a distance from the center of the generated threedimensional pseudo-volume data, cropping at a distance from thetool/catheter, or registering to previously obtained three dimensionalvolume data (CT) and using it to know which objects to filter.

Once the soft-tissue (such as a target lesion) is visible in the threedimensional reconstructed data (or in the two dimensional enhancedFluoroCT images), all three dimensional information in fluoroscopecoordinates can be obtained. When working with the raw three dimensionalreconstructed data the three dimensional information is obtaineddirectly from the data. Alternatively, when working with the twodimensional enhanced FluoroCT images, 2-angles markings are needed inorder to realize three dimensional positions (triangulation). In someembodiments, the three-dimensional reconstructed data is divided intoslices, which are thin enough such that triangulation is not required inorder to obtain three-dimensional position data.

The obtained three dimensional data may be utilized for real-timedetermination or confirmation of tool to soft-tissue target threedimensional relation. For example, the data may be used to determinewhether the tool reached the target, the orientation of the toolrelative to the target, the distance between the tool and the target, orwhether the target falls within an ablation zone of the tool.

Additionally, or alternatively, the three dimensional data may beutilized for correction of navigation. For example, the threedimensional positions can be transformed from fluoroscope coordinatesinto navigation system coordinates to improve navigation system accuracyat the region of interest. In one aspect, when utilized in an EMNsystem, fluoroscope coordinates can be transformed to antennacoordinates by assuming that the C-arm is perfectly perpendicular to theantenna and matching the catheter tip, seen in the fluoroscope video, tothe catheter position in antenna at the time the video was taken.Fluoroscopic to/from antenna registration can also be achieved bycomputing the angle between the C-arm and the antenna using the Earth'smagnetic field, attaching an EMN sensor to the fluoroscopic imagingdevice, or aligning a known two dimensional feature of the antenna.

Aspects of the present disclosure are described in detail with referenceto the figures wherein like reference numerals identify similar oridentical elements. As used herein, the term “distal” refers to theportion that is being described which is further from a user, while theterm “proximal” refers to the portion that is being described which iscloser to a user.

According to one aspect of the present disclosure, a system forconstructing fluoroscopic-based three dimensional volumetric data fromtwo dimensional fluoroscopic images is provided. The system includes acomputing device configured to facilitate navigation of a medical deviceto a target area within a patient and a fluoroscopic imaging deviceconfigured to acquire a fluoroscopic video of the target area about aplurality of angles relative to the target area. The computing device isconfigured to determine a pose of the fluoroscopic imaging device foreach frame of the fluoroscopic video and to construct fluoroscopic-basedthree-dimensional volumetric data of the target area in which softtissue objects are visible, in particular, using a fast iterativethree-dimensional construction algorithm. In aspects, the system isconfigured to determine the pose of the fluoroscopic imaging device byknowing the angle range of movement of the device and computing therelative rotational speed of the device along the range. In someaspects, the computing device is configured to determine a pose of thefluoroscopic imaging device for at least a plurality of frames of thefluoroscopic video via a structure of at least partially radio-opaquemarkers. In some aspects, the computing device is configured toconstruct fluoroscopic-based three dimensional volumetric data of thetarget area using a three dimensional construction algorithm, as knownin the art.

The medical device or tool may be a catheter assembly including anextended working channel configured to be positioned within a luminalnetwork of the patient or the medical device may be a radio-opaquemarker configured to be placed within the target area. The radio-opaquemarker is at least partially visible in the fluoroscopic video acquired.The medical device may be a surgical instrument used in a minimallyinvasive surgery, including robotic minimally invasive surgery.

The computing device may be further configured to create virtualfluoroscopic images of the patient from previously acquired CTvolumetric data and register the generated virtual fluoroscopic imageswith the acquired fluoroscopic video. The pose of the fluoroscopicimaging device for each frame of the fluoroscopic video may bedetermined based on the registration between the fluoroscopic video andthe virtual fluoroscopic images.

The computing device may further be configured to detect frames missingfrom the fluoroscopic video and supplement the detected missing frameswith corresponding virtual fluoroscopic images. The fluoroscopic-basedthree dimensional volumetric data may be constructed based on thefluoroscopic video and the corresponding virtual fluoroscopic images. Inone aspect, the fluoroscopic-based three dimensional volumetric data maybe registered with previously acquired CT data using image-basedtechniques such as “mutual information.” The fluoroscopic-based threedimensional volumetric data may be registered globally to the previouslyacquired CT data or locally, at the proximity of the target area ofinterest. A deep-learning based approach may be utilized in which thecomputing device “sees” many examples of suitable and non-suitableregistrations and learns how to register the very two differentmodalities.

Additionally, the computing device may be further configured to trackthe two dimensional position or orientation of the medical devicenavigated to the target region throughout the fluoroscopic video. Thecomputing device may be further configured to reconstruct positions ofthe medical device throughout the fluoroscopic video using astructure-from-motion technique. The pose of the fluoroscopic imagingdevice for each frame of the fluoroscopic video may be determined basedon the reconstructed positions. Additionally, or alternatively, the poseof the fluoroscopic imaging device for each frame of the fluoroscopicvideo may be determined based on an external angle measuring device. Theexternal angle measuring device may include an accelerometer, agyroscope, or a magnetic field sensor coupled to the fluoroscopicimaging device. Additionally, or alternatively, the pose of thefluoroscopic imaging device for each frame of the fluoroscopic video maybe determined via a structure of markers positioned externally to thepatient while capturing the images. Additionally, in aspects, thecomputing device may be configured to synchronize the captured frames ofthe target area and compensate for shifts in the fluoroscopic imagingdevice or patient movement to correct construction of thefluoroscopic-based three dimensional volumetric data. Additionally, oralternatively, the computing device may be configured to crop a regionof interest from the fluoroscopic-based three dimensional volumetricdata, project the cropped region of interest onto the captured frames,and sharpen or intensify at least one of the region of interest or thecaptured frame to identify soft tissue objects, or any other objects ofinterest.

In yet another aspect of the present disclosure a method forconstructing fluoroscopic-based three dimensional volumetric data fromtwo dimensional fluoroscopic images is provided. The method includesnavigating a medical device to a target area within a patient, acquiringa fluoroscopic video of the target area about a plurality of anglesrelative to the target area using a fluoroscopic imaging device,determining a pose of the fluoroscopic imaging device for each frame ofthe fluoroscopic video, and constructing fluoroscopic-based threedimensional volumetric data of the target area in which soft tissueobjects are visible using a fast iterative three dimensionalconstruction algorithm. The medical device may be a catheter assemblyincluding an extended working channel configured to be positioned withina luminal network of the patient or the medical device may be aradio-opaque marker configured to be placed within the target area. Theradio-opaque marker is at least partially visible in the fluoroscopicvideo acquired.

The method may further include creating virtual fluoroscopic images ofthe patient from previously acquired CT volumetric data, and registeringthe fluoroscopic video with the virtual fluoroscopic images, whereindetermining the pose of the fluoroscopic imaging device for each frameof the fluoroscopic video is based on the registration between thefluoroscopic video and the virtual fluoroscopic images. The method mayfurther include detecting frames missing from the fluoroscopic video andsupplementing the detected missing frames with corresponding virtualfluoroscopic images. Additionally, in aspects of the disclosure, themethod may further include tracking the two dimensional position ororientation of the medical device navigated to the target regionthroughout the fluoroscopic video.

The positions of the medical device throughout the fluoroscopic videomay be reconstructed using a structure-from-motion technique. The poseof the fluoroscopic imaging device for each frame of the fluoroscopicvideo may be determined based on the reconstructed positions.Additionally, or alternatively, the pose of the fluoroscopic imagingdevice for each frame of the fluoroscopic video may be determined basedon an external angle measuring device. The external angle measuringdevice may include an accelerometer, a gyroscope, or a magnetic fieldsensor coupled to the fluoroscopic imaging device. Additionally, oralternatively, the pose of the fluoroscopic imaging device for eachframe of the fluoroscopic video may be determined via a structure of atleast partially radio-opaque markers positioned with respect to thetarget area, externally to the patient's body, while capturing thefluoroscopic images. Additionally, in aspects, the method may includesynchronizing the captured frames of the target area and compensatingfor shifts in the fluoroscopic imaging device or patient movement tocorrect construction of the fluoroscopic-based three dimensionalvolumetric data. Additionally, or alternatively, the method may includecropping a region of interest from the fluoroscopic-based threedimensional volumetric data, projecting the cropped region of interestonto the captured frames, and sharpening or intensifying at least one ofthe region of interest or the captured frame to identify soft tissueobjects.

BRIEF DESCRIPTION OF THE DRAWINGS

Various aspects and embodiments of the present disclosure are describedhereinbelow with references to the drawings, wherein:

FIG. 1 is a perspective view of one illustrative embodiment of anelectromagnetic navigation (EMN) system incorporating a fluoroscopicimaging device in accordance with the present disclosure;

FIG. 2 illustrates a fluoroscopic imaging device model;

FIG. 3A is a flow chart of a method for constructing a three dimensionalvolume using a plurality of radio-opaque markers;

FIG. 3B is an illustration of an example of frames of a fluoroscopicvideo captured by a fluoroscopic imaging device showing markers and anextended working channel of a catheter assembly positioned within atarget region of a patient in accordance with the instant disclosure;

FIG. 4 is a flow chart of a method for constructing a three dimensionalvolume using either a single radio-opaque marker or the tip of anextended working channel of a catheter assembly;

FIG. 5 is a flow chart of a method for constructing a three dimensionalvolume using either a single radio-opaque marker or the tip of anextended working channel of a catheter assembly in conjunction with anangle measurement device;

FIG. 6 is a flow chart of a method for three dimensional modelconstruction in accordance with the instant disclosure;

FIG. 7 is an illustration of a frame of an original video captured by afluoroscopic imaging device, an image of the frame after being rampfiltered, and the resulting three dimensional volume;

FIG. 8 is an illustration of a three dimensional construction generatedaccording to a given angle range of fluoroscopic images/video;

FIG. 9A is a flow chart of another method for constructing fluoroscopicthree-dimensional volume via a structure of markers in accordance withthe present disclosure; and

FIG. 9B is a schematic illustration of a two-dimensional grid structureof sphere markers in accordance with the method of FIG. 9A.

DETAILED DESCRIPTION

The present disclosure is directed to a system and method forconstructing local three dimensional volumetric data, in which smallsoft-tissue objects are visible, from a video stream captured by astandard fluoroscopic imaging device available in most procedure rooms.The present disclosure is further directed to a system and method fordetermining a location of a medical device relative to a soft tissuetarget within a patient according to the three dimensional volumetricdata. The constructed fluoroscopic-based local three dimensionalvolumetric data or the location of the medical device relative to thesoft tissue target may be used for guidance, navigation planning,improved navigation accuracy, navigation confirmation, and treatmentconfirmation.

The terms “tool”, “surgery instrument”, “surgical device”, “energydevice”, “medical device” and alike may be used hereby interchangeably.

FIG. 1 depicts an Electromagnetic Navigation (EMN) system 100 configuredfor reviewing CT image data to identify one or more targets, planning apathway to an identified target (planning phase), navigating an extendedworking channel (EWC) 12 of a catheter assembly to a target (navigationphase) via a user interface, and confirming placement of the EWC 12relative to the target. One such EMN system is the ELECTROMAGNETICNAVIGATION BRONCHOSCOPY® system currently sold by Medtronic PLC. Thetarget may be tissue of interest identified by review of the CT imagedata during the planning phase. Following navigation, a medicalinstrument, such as a biopsy tool or other tool, may be inserted intothe EWC 12 to obtain a tissue sample from the tissue located at, orproximate to, the target.

As shown in FIG. 1, EWC 12 is part of a catheter guide assembly 40. Inpractice, the EWC 12 is inserted into bronchoscope 30 for access to aluminal network of the patient “P.” Specifically, EWC 12 of catheterguide assembly 40 may be inserted into a working channel of bronchoscope30 for navigation through a patient's luminal network. A locatable guide(LG) 32, including a sensor 44 is inserted into the EWC 12 and lockedinto position such that the sensor 44 extends a desired distance beyondthe distal tip of the EWC 12. The position and orientation of the sensor44 relative to the reference coordinate system, and thus the distalportion of the EWC 12, within an electromagnetic field can be derived.Catheter guide assemblies 40 are currently marketed and sold byMedtronic PLC under the brand names SUPERDIMENSION® Procedure Kits, orEDGE™ Procedure Kits, and are contemplated as useable with the presentdisclosure. For a more detailed description of the catheter guideassemblies 40, reference is made to commonly-owned U.S. PatentPublication No. 2014/0046315, filed on Mar. 15, 2013, by Ladtkow et al,U.S. Pat. No. 7,233,820, and U.S. Pat. No. 9,044,254, the entirecontents of each of which are hereby incorporated by reference.

EMN system 100 generally includes an operating table 20 configured tosupport a patient “P,” a bronchoscope 30 configured for insertionthrough the patient's “P's” mouth into the patient's “P's” airways;monitoring equipment 120 coupled to bronchoscope 30 (e.g., a videodisplay, for displaying the video images received from the video imagingsystem of bronchoscope 30); a tracking system 50 including a trackingmodule 52, a plurality of reference sensors 54 and a transmitter mat 56;and a computing device 125 including software and/or hardware used tofacilitate identification of a target, pathway planning to the target,navigation of a medical instrument to the target, and confirmation ofplacement of an EWC 12, or a suitable device therethrough, relative tothe target.

A fluoroscopic imaging device 110 capable of acquiring fluoroscopic orx-ray images or video of the patient “P” is also included in thisparticular aspect of system 100. The images, series of images, or videocaptured by the fluoroscopic imaging device 110 may be stored within thefluoroscopic imaging device 110 or transmitted to computing device 125for storage, processing, and display. Additionally, the fluoroscopicimaging device 110 may move relative to the patient “P” so that imagesmay be acquired from different angles or perspectives relative to thepatient “P” to create a fluoroscopic video. In one aspect of the presentdisclosure, fluoroscopic imaging device 110 includes an anglemeasurement device 111 which is configured to measure the angle of thefluoroscopic imaging device 110 relative to the patient “P.” Anglemeasurement device 111 may be an accelerometer. Fluoroscopic imagingdevice 110 may include a single imaging device or more than one imagingdevice. In embodiments including multiple imaging devices, each imagingdevice may be a different type of imaging device or the same type.Further details regarding the imaging device 110 are described in U.S.Pat. No. 8,565,858, which is incorporated by reference in its entiretyherein.

Computing device 125 may be any suitable computing device including aprocessor and storage medium, wherein the processor is capable ofexecuting instructions stored on the storage medium. The computingdevice 125 may further include a database configured to store patientdata, CT data sets including CT images, fluoroscopic data sets includingfluoroscopic images and video, navigation plans, and any other suchdata. Although not explicitly illustrated, the computing device 125 mayinclude inputs, or may otherwise be configured to receive, CT data sets,fluoroscopic images/video and other data described herein. Additionally,computing device 125 includes a display configured to display graphicaluser interfaces. Computing device 125 may be connected to one or morenetworks through which one or more databases may be accessed.

With respect to the planning phase, computing device 125 utilizespreviously acquired CT image data for generating and viewing a threedimensional model of the patient's “P's” airways, enables theidentification of a target on the three dimensional model(automatically, semi-automatically, or manually), and allows fordetermining a pathway through the patient's “P's” airways to tissuelocated at and around the target. More specifically, CT images acquiredfrom previous CT scans are processed and assembled into a threedimensional CT volume, which is then utilized to generate a threedimensional model of the patient's “P's” airways. The three dimensionalmodel may be displayed on a display associated with computing device125, or in any other suitable fashion. Using computing device 125,various views of the three dimensional model or enhanced two dimensionalimages generated from the three dimensional model are presented. Theenhanced two dimensional images may possess some three dimensionalcapabilities because they are generated from three dimensional data. Thethree dimensional model may be manipulated to facilitate identificationof target on the three dimensional model or two dimensional images, andselection of a suitable pathway through the patient's “P's” airways toaccess tissue located at the target can be made. Once selected, thepathway plan, three dimensional model, and images derived therefrom, canbe saved and exported to a navigation system for use during thenavigation phase(s). One such planning software is the ILOGIC® planningsuite currently sold by Medtronic PLC.

With respect to the navigation phase, a six degrees-of-freedomelectromagnetic tracking system 50, e.g., similar to those disclosed inU.S. Pat. Nos. 8,467,589, 6,188,355, and published PCT Application Nos.WO 00/10456 and WO 01/67035, the entire contents of each of which areincorporated herein by reference, or other suitable positioningmeasuring system, is utilized for performing registration of the imagesand the pathway for navigation, although other configurations are alsocontemplated. Tracking system 50 includes a tracking module 52, aplurality of reference sensors 54, and a transmitter mat 56. Trackingsystem 50 is configured for use with a locatable guide 32 andparticularly sensor 44. As described above, locatable guide 32 andsensor 44 are configured for insertion through an EWC 12 into apatient's “P's” airways (either with or without bronchoscope 30) and areselectively lockable relative to one another via a locking mechanism.

Transmitter mat 56 is positioned beneath patient “P.” Transmitter mat 56generates an electromagnetic field around at least a portion of thepatient “P” within which the position of a plurality of referencesensors 54 and the sensor element 44 can be determined with use of atracking module 52. One or more of reference sensors 54 are attached tothe chest of the patient “P.” The six degrees of freedom coordinates ofreference sensors 54 are sent to computing device 125 (which includesthe appropriate software) where they are used to calculate a patientcoordinate frame of reference. Registration, as detailed below, isgenerally performed to coordinate locations of the three dimensionalmodel and two dimensional images from the planning phase with thepatient's “P's” airways as observed through the bronchoscope 30, andallow for the navigation phase to be undertaken with precise knowledgeof the location of the sensor 44, even in portions of the airway wherethe bronchoscope 30 cannot reach. Further details of such a registrationtechnique and their implementation in luminal navigation can be found inU.S. Patent Application Pub. No. 2011/0085720, the entire content ofwhich is incorporated herein by reference, although other suitabletechniques are also contemplated.

Registration of the patient's “P's” location on the transmitter mat 56is performed by moving LG 32 through the airways of the patient's “P.”More specifically, data pertaining to locations of sensor 44, whilelocatable guide 32 is moving through the airways, is recorded usingtransmitter mat 56, reference sensors 54, and tracking module 52. Ashape resulting from this location data is compared to an interiorgeometry of passages of the three dimensional model generated in theplanning phase, and a location correlation between the shape and thethree dimensional model based on the comparison is determined, e.g.,utilizing the software on computing device 125. In addition, thesoftware identifies non-tissue space (e.g., air filled cavities) in thethree dimensional model. The software aligns, or registers, an imagerepresenting a location of sensor 44 with a the three dimensional modeland two dimensional images generated from the three dimension model,which are based on the recorded location data and an assumption thatlocatable guide 32 remains located in non-tissue space in the patient's“P's” airways. Alternatively, a manual registration technique may beemployed by navigating the bronchoscope 30 with the sensor 44 topre-specified locations in the lungs of the patient “P”, and manuallycorrelating the images from the bronchoscope to the model data of thethree dimensional model.

Following registration of the patient “P” to the image data and pathwayplan, a user interface is displayed in the navigation software whichsets for the pathway that the clinician is to follow to reach thetarget. One such navigation software is the ILOGIC® navigation suitecurrently sold by Medtronic PLC.

Once EWC 12 has been successfully navigated proximate the target asdepicted on the user interface, the locatable guide 32 may be unlockedfrom EWC 12 and removed, leaving EWC 12 in place as a guide channel forguiding medical instruments including without limitation, opticalsystems, ultrasound probes, marker placement tools, biopsy tools,ablation tools (i.e., microwave ablation devices), laser probes,cryogenic probes, sensor probes, and aspirating needles to the target.

Having described the components of system 100 depicted in FIG. 1, thefollowing description of FIGS. 2-10 provides an exemplary workflow ofusing the components of system 100, including the fluoroscopic imagingdevice 110, to construct local three dimensional volumetric data of adesired region of interest and to determine medical device tosoft-tissue target relation using the fluoroscopic imaging device 110 ofsystem 100. The systems and methods described herein may be useful forvisualizing a particular target region of a patient, and furthermore,for visualizing medical devices relative to the target during a medicalprocedure, utilizing imaging devices which are commonly located within asurgical setting during EMN procedures, thereby obviating the need forsubsequent Mill or CT scans.

Turning now to FIG. 2, a fluoroscopic imaging device 110 model isillustrated. The fluoroscopic imaging device 110 includes an X-raysource 201 and a detector 203. The detector 203 defines a plurality oftwo dimensional pixels p. Each two dimensional pixel P_(i) is associatedwith a single X-ray beam l_(i), traversing three dimensional space fromthe X-ray source 201 to the detector 203. The detector 203 size D andthe source-detector distance SDD determine the Field-of-View angle usingthe following formula:

$\theta = {2\; {{\tan^{- 1}\left( \frac{D}{2\; {SDD}} \right)}.}}$

Different fluoroscopes differ primarily by detector size andsource-detector distance. In the fluoroscopic data, pixels are notnormalized to a specific scale. Their brightness depends on thegain/exposure used by the fluoroscope and on other objects in the scenewhich the X-rays traverse between the source and the detector such asthe table, the surgical device, etc. In the CT data, each voxel ismeasured in Hounsfield units. Hounsfield units are measured with respectto the brightness observed of water and air using the following formula:

${HU} = {1000{\frac{\mu - \mu_{water}}{\mu_{water} - \mu_{air}}.}}$

μ are attenuation coefficients. They range from 0 to infinity andmeasure how difficult it is for an X-ray beam l_(i) to traverse throughmatter of the three dimensional volume 205. “Thicker” matter has alarger μ. HU scales attenuation coefficients such that air is placed at−1000, water at 0 and thicker matter goes up to infinity.

Using the Beer-Lambert law, each two dimensional pixel p_(i) in thefluoroscope's detector 203 is given by:

p _(ij) =I ₀exp(−∫_(l) _(i,j) μ(l)dl).

μ(x, y, z) is the attenuation coefficient of the three dimensionalvolume 205 at position (x, y, z). I₀ is the X-ray source 201 energy(higher energy produces brighter image). For simplicity, assume I₀=1.Taking the logarithm, each two dimensional pixel p_(i) is representedby:

log(p _(i,j))=−∫_(l) _(i,j) μ(l)dl.

This is true for each two dimensional detector pixel p_(i) whichprovides a linear equation on the three dimensional attenuationcoefficients. If many such equations are utilized (to solve for each twodimensional pixel p_(i) of the detector 203) then the attenuationcoefficients may be solved and the three dimensional volume data of thethree dimensional volume 205 can be constructed.

Fluoroscope to CT—Discretization:

The three dimensional volume 205 can be divided into a discrete grid,with voxels sitting at (x_(k), y_(k), z_(k)). Thus, the equation forsolving each two dimensional pixel p_(i) can then be written as:

log(p _(i,j))=−Σ_(k) w _(k) ^(i,j)μ(x _(k) ,y _(k) ,z _(k)).

The left hand side of the equation above is the observed two dimensionalpixel p_(i) of the detector 203 and the right hand side of the equationabove is a weighted sum of the attenuation coefficients (that is to besolved) with w_(k) ^(i,j) which are determined by the known fluoroscopicimaging device position of the X-ray source 201.

In order to be able to solve for the volumetric attenuation coefficientvalues enough linear equations are needed, that is, enough twodimensional observations of different two dimensional pixels p arerequired. Standard detectors usually have 1024×1024 pixels, where eachpixel is represented by a single equation. Therefore, many twodimensional observations are required to be solved for (many twodimensional observed pixels) to reconstruct a three dimensional volume(to solve for many voxels). That is many two dimensional pixels arerequired to solve for the many voxels. In order for the weights in theequations to be known, the fluoroscopic imaging device X-ray source 201configuration (position, orientation, field of view) must be known.

In use, the three dimensional volume 205 is a portion of a patient'sbody. A fluoroscopic video which is comprised of multiple fluoroscopeimages (as frames of the video) are taken from many different locationsrelative to the patient, for example a 180° rotation about the patient,to acquire multiple observations of two dimensional pixels p atdifferent positions relative to the patient. As will be described ingreater detail below, the position of the fluoroscopic imaging devicerelative to the patient at a given time can be determined using multipletechniques, including structure-from-motion analysis of radio-opaquemarkers placed within the patient (FIGS. 3A-3B), a registration betweenthe captured fluoroscopic images/video and generated virtualfluoroscopic images (FIG. 4), an external angle measurement device suchas an accelerometer, gyroscope, or magnetic field sensor (FIG. 5), or byan external structure of markers (FIGS. 9A-9B). In one aspect, thesystem may correct for patient movements when measuring angles when amarker is utilized which moves with the movement of the body.Specifically, all of the two dimensional fluoroscopic images may besynched to the same three dimensional position based on the position ofthe placed marker in each of the images.

Turning now to FIGS. 3A-3B and 4-6, methods for constructing a localthree dimensional volume of a target region using a standardfluoroscopic imaging device (such as the fluoroscopic imaging device 110of FIG. 1) in conjunction with a system such as the system described inFIG. 1 will now be described with particular detail. Although themethods illustrated and described herein are illustrated and describedas being in a particular order and requiring particular steps, any ofthe methods may include some or all of the steps and may be implementedin any order not specifically described.

The fluoroscopic-based three dimensional volume generated by any of themethods described below can be incorporated into system 100 for multiplepurposes. For example, the fluoroscopic-based three dimensional volumecan be registered with the previously generated three dimensionalvolumetric data that was utilized for navigation of the medicalinstrument. The system 100 may utilize the registration between thefluoroscopic-based three dimensional volume and the previously acquiredthree dimensional volumetric data to update the calculated position ofthe sensor 44 (FIG. 1) which has been placed in the body of the patient.More particularly, the three dimensional model of a patient's lungs,generated from previously acquired CT scans, may not provide a basissufficient for accurate guiding of medical instruments to a targetduring an electromagnetic navigation procedure. In certain instances,the inaccuracy is caused by deformation of the patient's lungs duringthe procedure relative to the lungs at the time of the acquisition ofthe previously acquired CT data. This deformation (CT-to-Bodydivergence) may be caused by many different factors, for example:sedation vs. no sedation, bronchoscope changing patient pose and alsopushing the tissue, different lung volume because CT was in inhale whilenavigation is during breathing, different bed, day, etc. Thus, anotherimaging modality is necessary to visualize targets and/or a terminalbronchial branch, and enhance the electromagnetic navigation procedureby correcting the navigation during the procedure, enablingvisualization of the target, and confirming placement of the surgicaldevice during the procedure. For this purpose, the system describedherein processes and converts image data captured by the fluoroscopicimaging device 110, as will be described in detail below. Thisfluoroscopic image data may be utilized to identify such targets andterminal bronchial branches or be incorporated into, and used to update,the data from the CT scans in an effort to provide a moreaccurate/correction of the electromagnetic navigation procedure.

Additionally, users may visually confirm that the placement of thenavigated medical instrument is positioned in a desired locationrelative to a target tissue within a target area. Additionally, thefluoroscopic-based three dimensional volume can be utilized to visualizethe target area in three dimensions after a procedure is performed. Forexample, the fluoroscopic-based three dimensional volume can be utilizedto visualize the target area after markers are placed within the targetarea, after a biopsy is taken, or after a target is treated.

With particular reference to FIGS. 3A-3B, a method for constructing athree dimensional volume using a plurality of radio-opaque markersplaced proximate the target will now be described and referred to asmethod 300. Method 300 begins at step 301 where a marker placementdevice is navigated to a target area utilizing an electromagneticnavigation system, such as the EMN system 100 (FIG. 1) described above.The navigation of the marker placement device to the target area may beaccomplished using a previously created navigation plan which includesroutes created during the planning phase. In step 303, radio-opaquemarkers are placed within the target area. In one example, fourradio-opaque markers are utilized. However, less than four or more thanfour radio-opaque markers may be used.

In step 305, with the radio-opaque markers placed in the target area,the fluoroscopic imaging device is positioned such that all of theradio-opaque markers placed in step 303 are visible. That is, step 305includes aligning the fluoroscopic imaging device such that it canrotate 30° around the markers with all of the markers visible. In step307, the fluoroscopic imaging device is used to capture a video of abouta 30° rotation of the imaging device 110 about the patient, and thusaround the markers (rotation from −15° to +15°). By rotating up to 15°(on each side) from the centered angle, it can be ensured that themarkers will remain in the images/frames for the entire rotation videoand that the imaging device will not hit the patient or the bed. FIG. 3Billustrates six frames f1-f6 of the captured video. Each of frames f1-f6is an image of the fluoroscopic video showing the different position andorientation of each radio-opaque marker m1-m4 at different points intime of the video, where the fluoroscopic imaging device is positionedat a different angle relative to the patient at each given time.

In step 309, the two-dimensional position of each radio-opaque marker istracked throughout the entire video. In step 311, the marker positionsare constructed in three dimensional using structure-from-motiontechniques and the pose of the fluoroscopic imaging device is obtainedfor each video frame. Structure-from-motion is a method forreconstructing three dimensional positions of points, along withfluoroscopic imaging device locations (camera poses) by tracking thesepoints in a two dimensional continuous video. In step 311, byintroducing markers into the patient, the positions and orientations ofthose markers can be tracked along a continuous fluoroscope rotationvideo, their three dimensional position in space can be reconstructed,and the corresponding fluoroscopic imaging device locations can bedetermined. With the fluoroscopic imaging device locations determinedthrough analysis of the video, the fluoroscopic imaging device locationscan be used to solve for the three dimensional data.

In step 313, a local three dimensional volume is constructed.Specifically, a fast iterative reconstruction algorithm (FIG. 6) is usedto reconstruct a local three dimensional volume at the area of themarkers which correspond to the local anatomy and which soft-tissues arevisible. Step 313 may include reconstructing a global three dimensionalvolume from the acquired two dimensional fluoroscopic data and croppingthe global three dimensional volume at the area of the target to createa “FluoroCT Blob” volume. This cropped volume may be displayed to theuser in the form of raw three dimensional data or as two dimensionalreprojected images. In this cropped volume, all anatomy in the targetarea, including the target tissue, will be visible. The reprojectedimage can be intensified (which does not include the distant denseobscuring objects) by stretching the darker value to be black and thebrighter value to be white to increase differentiation and also sharpenin either three dimension before projection or in the projected images,such that the soft tissue is visually identified.

As described above, the fluoroscopic-based three dimensional volume canbe incorporated into system 100 for multiple purposes. For example, thefluoroscopic-based three dimensional volume can be registered with thepreviously generated three dimensional volumetric data that was utilizedfor navigation of the medical instrument. The system 100 may utilize theregistration between the fluoroscopic-based three dimensional volume andthe previously acquired three dimensional volumetric data to update thecalculated position of the sensor 44 (FIG. 1). Additionally, users mayvisually confirm that the placement of the navigated medical instrumentis positioned in a desired location relative to a target tissue within atarget area.

Turning now to FIG. 4, a method for constructing a three dimensionalvolume using either a single radio-opaque marker placed proximate thetarget or the distal portion of a navigated tool, such as the tip of theextended working channel of the catheter assembly positioned proximatethe target will now be described and referred to as method 400. Althoughdescribed as utilizing the tip of the extended working channel of thecatheter assembly, method 400 may utilize any tool to achieve thisfunction. For example, the tip of a navigated catheter, the tip of abiopsy tool, or the tip of a treatment tool may be utilized. In oneaspect, the tool is navigated transbronchially to the target. In otheraspects, the tool may be of a tool inserted into a patientpercutaneously, for example, a transthoracic navigation of a treatmentdevice such as an ablation device.

Method 400 begins at step 401 where the extended working channel isnavigated to a target area utilizing an electromagnetic navigationsystem, such as the EMN system 100 (FIG. 1) described above. Thenavigation of the EWC to the target area is accomplished using apreviously created navigation plan which includes routes created duringthe planning phase. Method 400 may optionally include the additionalstep of navigating a marker placement device, via the EWC, to the targetarea to place a single radio-opaque marker within the region of thetarget (step 403). In one aspect, step 401 includes percutaneouslyinserting a tool to the target area.

After the EWC or tool is in position, or after the radio-opaque markeris placed, the fluoroscopic imaging device is positioned such that thenavigated tip of the EWC or tool (and/or the placed radio-opaque marker)is visible within the field of view of the fluoroscopic imaging device.That is, step 405 includes aligning the fluoroscopic imaging device suchthat it can rotate 30° around the marker with the marker visible and/oraround the tip of the EWC or tool with the tip of the EWC or toolvisible. In step 407, the fluoroscopic imaging device is used to capturea video of about a 30° rotation of the imaging device 110 about thepatient, and thus around the marker and/or tip of the EWC or tool(rotation from −15° to +15°). By rotating up to 15° (on each side) fromthe centered angle, it can be ensured that the marker and/or tip of theEWC or tool will remain in the images/frames for the entire rotationvideo and that the imaging device will not hit the patient or the bed.Step 407 may include capturing a video of about a 30° rotation aroundthe distal portion of the EWC (and the radio-opaque marker, if placed).If a 30° rotation video is captured, then one angle (in the middle ofthe range) is enough. That is, two projections with 30° between them areenough to confirm or correct three dimensional relation of tools to softtissue.

In step 409, the two-dimensional position of the distal portion of theEWC or tool (and/or the radio-opaque marker, if placed) is trackedthroughout the entire captured video.

In step 411, virtual fluoroscopic images are created from previouslyacquired CT data. The previously acquired CT data is typically the CTdata used during the planning phase to plan a navigation path to thetarget. In step 411, the CT data is manipulated to create a computermodel of fluoroscopic images of the patient. The location of the targetin the virtual fluoroscopic images corresponds to the location of thetarget identified by the clinician during the planning phase. Thevirtual fluoroscopic images generated by the system, based off of thepreviously acquired CT data, depict the field of view that would becaptured by a fluoroscopic imaging device. Additionally, each of thevirtual fluoroscopic images has a virtual fluoroscopic imaging devicepose.

In step 413, each video frame of the fluoroscopic video captured in step407 is registered to the previously acquired CT data by matching each ofthe fluoroscopic video frames to the virtual fluoroscopic images. Instep 415, the fluoroscopic imaging device pose of each video frame ofthe captured fluoroscopic video is determined based on the registrationof step 413. That is, once a fluoroscopic frame is matched to a virtualfluoroscopic image, the virtual fluoroscopic imaging device pose of thevirtual fluoroscopic image can be associated with the correspondingfluoroscopic frame.

In step 417, the origin of the fluoroscopic imaging device posedetermined in step 415 is corrected by using the tracked position of thedistal portion of the EWC or tool (and/or the radio-opaque marker, ifplaced), which is used to compensate for movement of the patient, suchas movement caused by breathing. In step 419, a local three dimensionalvolume is constructed. Specifically, a fast iterative reconstructionalgorithm (FIG. 6) is used to reconstruct a local three dimensionalvolume at the area of the target lesion which corresponds to the localanatomy and which soft-tissues are visible. Step 319 may includereconstructing a global three dimensional volume from the acquired twodimensional fluoroscopic data and cropping the global three dimensionalvolume at the area of the target to create a “FluoroCT Blob” volume.This cropped volume may be displayed to the user in the form of rawthree dimensional data or as two dimensional reprojected images. In thiscropped volume, all anatomy in the target area, including the targettissue, will be visible. The reprojected image can be intensified (whichdoes not include the distant dense obscuring objects) by stretching thedarker value to be black and the brighter value to be white to increasedifferentiation and also sharpen in either three dimension beforeprojection or in the projected images, such that the soft tissue isvisually identified.

Method 400 may also include the additional step (step 421) of completingthe fluoroscopic video captured in step 407 to include virtualfluoroscopic images that are generated by the system, which arerepresentative of fluoroscopic imaging device poses that are outside therange of fluoroscopic imaging device poses captured in the fluoroscopicvideo. Specifically, in aspects, the previously generated CT volumetricdata of the patient which is used to create a navigation plan may alsobe utilized by system 100 to generate virtual fluoroscopic images of thepatient. The generated virtual fluoroscopic images are fluoroscopic-likeimages which display a view to the user of what a fluoroscopic image ofa patient should look like if captured at a given angle by afluoroscopic imaging device. In step 421, the virtual fluoroscopicimages may be used to fill any gaps in the captured fluoroscopic video(captured in step 407). This may include, for example, replacement ofimages, such as frames, of the captured video that are skewed ordamaged. Additionally, or alternatively, this may include, for example,supplementing the captured fluoroscopic video (captured in step 407)with virtual fluoroscopic images that are representative of fluoroscopicimages that are outside the range of angles included in the fluoroscopicvideo. For example, if the fluoroscopic video included a sweep of abouta 30° range about the patient, virtual fluoroscopic images that areoutside the 30° range could be incorporated into the video to generate afluoroscopic video that has a range of greater than 30°.

Method 300 (FIG. 3) and method 400 (FIG. 400) are both used to constructthree dimensional CT volumetric data using fluoroscopic video withoutknowing the fluoroscopic imaging device poses of each of the frames ofthe fluoroscopic video. To this end, each of methods 300 and 400 requiresteps of determining the fluoroscopic imaging device pose of each of theframes of the fluoroscopic video utilizing image-based techniques. Incontrast, and as described in greater detail below, method 500 (FIG. 5)is a method for constructing three dimensional CT volumetric data wherethe fluoroscopic imaging device pose of each of the frames of theacquired fluoroscopic video are determined using a pose/anglemeasurement device, which may include an accelerometer, a gyroscope, ormagnetic field detector to detect the position/pose of the fluoroscopicimaging device relative to the patient.

Method 500 is a method for constructing a three dimensional volume usingeither a single radio-opaque marker placed proximate the target or thetip of the extended working channel of the catheter assembly positionedproximate the target, in conjunction with a fluoroscope anglemeasurement device. Method 500 begins at step 501 where fluoroscopecalibration is performed using a fluoroscope calibration jig to computethe canonical fluoroscope projection parameters and geometry. This isdone once per fluoroscope device, in a setup phase by a technician. Thecalibration jig is used to determine, in an automated process, both theprojection parameters of the fluoroscope (field of view angle) as wellas the geometry of the C-arm: position relative to rotation axis. Theseparameters are sometimes given in technical drawings per fluoroscopedevice, but may also be found using our calibration jig. In step 503,the previously acquired CT volumetric data is imported into the systemalong with the previously generated navigation plan.

In step 505, the extended working channel is navigated to a target areautilizing an electromagnetic navigation technique using a system such asthe EMN system 100 (FIG. 1) described above. The navigation of the EWCto the target area is accomplished using the previously creatednavigation plan which includes routes created during the planning phase.Method 500 may optionally include the additional step (step 507) ofnavigating a marker placement device, via the EWC, to the target area toplace a single radio-opaque marker within the region of the target.

After the EWC or tool is in position, or after the radio-opaque markeris placed, method 500 proceeds to step 509 which includes aligning thefluoroscopic imaging device such that it can rotate 30° around themarker with the marker visible and/or the tip of the EWC or the toolsuch that the tip of the EWC or tool is visible. In step 511, thefluoroscopic imaging device is used to capture a video of about a 30°rotation of the imaging device 110 about the patient, and thus aroundthe marker or tip of the EWC or tool (rotation from −15° to +15°). Byrotating up to 15° (on each side) from the centered angle, it can beensured that the marker or tip of the EWC or tool will remain in theimages/frames for the entire rotation video and that the imaging devicewill not hit the patient or the bed. If a 30° rotation video iscaptured, then one angle (in the middle of the range) is enough. Thatis, two projections with 30° between them are enough to confirm orcorrect three dimensional relation of tools to soft tissue. In step 513,the two-dimensional position and orientation of the distal end of theEWC (and/or the radio-opaque marker, if placed) is tracked throughoutthe entire captured video.

In step 515, the calibration data from step 501 is utilized incombination with measurements from an external angle measurement deviceto compute the fluoroscopic imaging device location (origin-less) inworld coordinates. Specifically, the calibration jig is used, asdescribed above, to automatically find the projection parameters and theC-arm geometry of the specific fluoroscope device by using optimizationmethods. Once these parameters are known, the angle taken from the anglemeasurement device, that is, the angle of the detector of thefluoroscope, determines a unique three dimensional pose for thedetector. It cannot be located in any other place in space other thanthe once explained by the given angle and the setup parameters.

In step 517, a local three dimensional volume is constructed.Specifically, a fast iterative reconstruction algorithm (FIG. 6) is usedto reconstruct a local three dimensional volume at the area of thetarget tissue which corresponds to the local anatomy and whichsoft-tissues are visible. Step 517 may include reconstructing a globalthree dimensional volume from the acquired two dimensional fluoroscopicdata and cropping the global three dimensional volume at the area of thetarget to create a “FluoroCT Blob” volume. This cropped volume may bedisplayed to the user in the form of raw three dimensional data or astwo dimensional reprojected images. In this cropped volume, all anatomyin the target area, including the target tissue, will be visible. Thereprojected image can be intensified (which does not include the distantdense obscuring objects) by stretching the darker value to be black andthe brighter value to be white to increase differentiation and alsosharpen in either three dimension before projection or in the projectedimages, such that the soft tissue is visually identified.

CT reconstruction methods will now be described. CT reconstructionmethods can be divided into Analytic methods (Radon, FDK . . . ) andAlgebraic methods (ART, SART . . . ). Analytic methods assume a veryspecific configuration, such as a full 180° rotation, and reconstructthe CT volume in a single iteration (using some exact formulas).Algebraic methods are more flexible but slower and treat the problem asa large equation system which is solved iteratively (using some gradientdescent method). All methods use projection (three dimensional to twodimensional) and back-projection (two dimensional to three dimensional).

With respect to projection, since each detector pixel is essentially aweighted sum of three dimensional voxels along a ray, the detector imagecan be viewed as a two dimensional projection of the three dimensionalvolume from some fluoroscopic imaging device location. If the threedimensional volume data is already available, then the fluoroscopeimages can be reproduced by projecting it from the known fluoroscopicimaging device locations. A three dimensional reconstruction isconsidered good if its two dimensional projections resemble the observedfluoroscope images it was created from.

With respect to back-projection, at each voxel, the back-projectiondetermines which rays traversed through a particular voxel and sums themtogether. In order to make this determination, the fluoroscopic imagingdevice locations must be known. If the back-projection operator wereapplied to the fluoroscope images of the captured video, then a threedimensional volume could be constructed, but the constructed threedimensional volume would be very blurry and inaccurate because while thetrue center voxel is summed many times, many irrelevant voxelssurrounding the true center voxel are also summed many times. Afterreconstructing a three dimensional volume using one method or another,the quality of the reconstruction is evaluated by taking thereconstructed three dimensional data, projecting it into two dimensionalframes (which may be virtual-fluoroscopic frames) and comparing thosetwo dimensional frames to the original fluoroscopic two dimensionalframes from which the three dimensional data was created. A goal of thevolume reconstruction algorithm is to find three dimensional data whichexplains the two dimensional observations, such that if the threedimensional data were to be projected back into two dimensional frames,those frames would look like the original, real, fluoroscopic frames.When the product is blurry, it means that the projections are blurry anddo not match the real images. In order to address this issue, the methodprovides for a ramp-filter and correction iteration.

Turning now to FIG. 6, a method for reconstructing a local threedimensional volume utilizing a fast iterative algorithm will now bedescribed and referred to as method 600. The three dimensional volumereconstruction algorithm (for example method 600) consists of multipleiterations of projection—back-projection. The goal of the algorithm isto find three dimensional data which explains the two dimensionalobservations of the fluoroscopic imaging device. If it succeeds infinding such data then the three dimensional data found is assumed to bethe three dimensional anatomy of the patient. In each iteration thecurrent three dimensional data found is projected into two dimensions,which should look like the original fluoroscope video, then the twodimensional errors are backprojected back into the three dimensionaldata, and in such a manner the three dimensional data is updated. Thisis repeated several times until the three dimensional data converges andthe process stops. In order to speed up the process, some filtering isdone to the two dimensional projected images before backprojecting themback into three dimensions. The ramp-filter is just an example filterwhich proves to be efficient in speeding up the convergence process.This filter is applied when reconstructing standard CTs with the classicRadon transform. With the classic approach, this process is done in asingle iteration: Filtering (ramp-filter for example), Back-projection.In this method, this is repeated iteratively in several steps.

Method 600 begins in step 601 where the equation begins with an initialvolume V (can just be zeros). In step 603, the volume V is projectedfrom known fluoroscopic imaging device locations into images Q. Forexample, projection may be done, not of the fluoroscopic images, but ofthe fluoroscopic images filtered by a ramp function. The iterations withresiduals, described below, may undergo a ramp-filter beforeback-projections. In step 605, the residuals R_(i)=P_(i)Q_(i) arecomputed where P_(i) are the observed projections from the capturedfluoroscope video. In step 606, R_(i) is convolved with the ramp-filteras in Radon. In step 607, it is determined whether R_(i) is below apredetermined threshold. If R_(i) is below the predetermined threshold(yes in step 607), then method 600 is complete. If R_(i) is not belowthe predetermined threshold (no in step 607), then method 600 proceedsto step 609. In step 609, R_(i) is back-projected into E, the correctionvolume. In step 611, volume V is set to V+E (V=V+E) and method 600reverts to step 603 where the volume V (now V+E) is projected from knownfluoroscopic imaging device locations into images Q.

Turning now to FIGS. 7 and 8. FIG. 7 illustrates a frame of an originalvideo captured by a fluoroscopic imaging device 700, an image of theframe after being ramp-filtered 703, and an image of the resulting threedimensional volume 705. FIG. 8 is an illustration of a three dimensionalconstruction at angles 801-88, where 801 is 30 degrees, 803 is 60degrees, 805 is 90 degrees, 807 is 120 degrees, 809 is 150 degrees, and811 is 180 degrees.

Reference is now made to FIG. 9A, which is a flow chart of anothermethod for constructing fluoroscopic three-dimensional volume via astructure of markers in accordance with the present disclosure. A methodfor constructing fluoroscopic-based three-dimensional volumetric data ofa target area within a patient from two-dimensional fluoroscopic images,is hereby disclosed. In a step 900, a sequence of images of the targetarea and of a structure of markers is acquired via a fluoroscopicimaging device. The structure of markers may include a plurality of atleast partially radio-opaque markers arranged in a certain pattern. Thestructure of markers is positioned externally to the patient, e.g.,under the patient, while capturing the images. The structure of markersis further positioned such that each image includes a projection of atleast a portion of the structure of markers. The structure of markersmay be positioned between the patient and the fluoroscopic imagingdevice. In some embodiments, the structure of markers may be of atwo-dimensional pattern. In some embodiments, the structure of markersmay be of a periodic pattern, such as a grid. In some embodiments, thetarget is a soft-tissue target. In some embodiments, the target area mayinclude, for example, at least a portion of the lungs, and asexemplified with respect to the system of FIG. 1.

In a step 910, a pose of the fluoroscopic imaging device for at least aplurality of images of the sequence of images may be estimated. The poseestimation may be performed based on detection of a possible and mostprobable projection of the structure of markers, as a whole, on eachimage of the plurality of images. A probability map may be thengenerated for each image indicating the probability of each pixel in theimage to be a marker of the structure of markers. Multiple virtualcandidate for the projection of the structure of markers on the imagemay be generated by virtually positioning the fluoroscope in possibledifferent locations, including possible orientations (i.e., differentposes). The candidate having the highest probability of being theprojection of the structure of markers on the image may be thenidentified based on the probability map. The virtual pose of thefluoroscope associated with the identified candidate may be thendetermined as the estimated pose of the fluoroscope while capturing theimage. Optionally, the process of identifying a candidate may berefined. A locally deformed version of the candidate may be generatedbased on the probability map in order to maximize its probability ofbeing the projection of the structure of markers on the image. A newvirtual candidate may be then fitted to the locally deformed version ofthe identified candidate. The virtual pose of the fluoroscope whichwould generate the new improved candidate is the calculated anddetermined as the estimated pose of the fluoroscope while capturing theimage. Further details with respect to the disclosed pose estimation maybe found in commonly-owned U.S. patent application Ser. No. 62/628,017,entitled: “SYSTEM AND METHOD FOR POSE ESTIMATION OF AN IMAGING DEVICEAND FOR DETERMINING THE LOCATION OF A MEDICAL DEVICE WITH RESPECT TO ATARGET”, filed on Feb. 8, 2018, by Barak et al., the entire content ofwhich is hereby incorporated by reference.

In a step 920, a fluoroscopic-based three-dimensional volumetric data ofthe target area may be constructed based on the estimated poses of thefluoroscopic imaging device according to the disclosed systems andmethods. However, other systems and methods, as known in the art, whichare based on the imaging device pose information may be used.

In an optional step 930, a medical device may be positioned in thetarget area prior to the acquiring of the sequence of images. Thus, thesequence of images and consequently the fluoroscopic-basedthree-dimensional volumetric data may also include a projection of themedical device in addition to the target. The offset (i.e., Δx, Δy andΔz) between the medical device and the target may be then determinedbased on the fluoroscopic-based three-dimensional volumetric data. Thetarget may be visible or better exhibited in the generatedthree-dimensional volumetric data. Therefore, the target may bedetected, automatically, or manually by the user, in thethree-dimensional volumetric data. The medical device may be detected,automatically or manually by a user, in the sequence of images, ascaptured, or in the generated three-dimensional volumetric data. Theautomatic detection of the target and/or the medical device may beperformed based on systems and methods as known in the art and such asdescribed, for example, in commonly-owned U.S. patent application Ser.No. 62/627,911, entitled: “SYSTEM AND METHOD FOR CATHETER DETECTION INFLUOROSCOPIC IMAGES AND UPDATING DISPLAYED POSITION OF CATHETER”, filedon Feb. 8, 2018, by Birenbaum et al. The manual detection may beperformed by displaying to the user the three-dimensional volumetricdata and/or captured images and requesting his input. Once the targetand the medical device are detected in the three-dimensional volumetricdata and/or the captures images, their location in the fluoroscopiccoordinate system of reference may be obtained and the offset betweenthem may be determined.

The offset between the target and the medical device may be utilized forvarious medical purposes, including facilitating approach of the medicaldevice to the target area and treatment. The navigation of a medicaldevice to the target area may be facilitated via a tracking or locatingsystem and a display, such as tracking system 50 and monitoringequipment 120 of FIG. 1. The locating system locates or tracks themotion of the medical device through the patient's body. The display maydisplay the medical device location to the user with respect to thesurroundings of the medical device within the patient's body and thetarget. The locating system may be, for example, an electromagnetic oroptic locating system, or any other such system as known in the art.When, for example, the target area includes a portion of the lungs, themedical device may be navigated to the target area through the airwaysluminal network and as described with respect to FIG. 1.

In an optional step 940, a display of the location of the medical devicewith respect to the target may be corrected based on the determinedoffset between the medical device and the target. In some embodiments, a3D rendering of the target area may be displayed on the display. The 3Drendering of the target area may be generated based on CT volumetricdata of the target area which was acquired previously, i.e., prior tothe current procedure or operation (i.e., preoperative CT). In someembodiments, the locating system may be registered to the 3D renderingof the target, such as described, for example, with respect to FIG. 1.The correction of the offset between the medical device and the targetmay be then performed by updating the registration of the locatingsystem to the 3D rendering. Generally, to perform such updating, atransformation between coordinate system of reference of thefluoroscopic images and the coordinate system of reference of thelocating system should be known. The geometrical positioning of thestructure of markers with respect to the locating system may determinesuch a transformation. In some embodiments the structure of markers andthe locating system may be positioned such that the same coordinatesystem of reference would apply to both, or such that the one would beonly a translated version of the other. For example, and with referenceto FIG. 1, transmitter mat 56 may be incorporated with the structure ofmarkers.

In some embodiments, the updating of the registration of the locatingsystem to the 3D rendering (e.g., CT-based) may be performed in a localmanner and/or in a gradual manner. For example, the registration may beupdate only in the surroundings of the target, e.g., only within acertain distance from the target. This is since the update may be lessaccurate when not performed around the target. In some embodiments, theupdating may be performed in a gradual manner, e.g., by applying weightsaccording to distance from the target. In addition to accuracyconsiderations, such gradual updating may be more convenient or easierfor the user to look at, process and make the necessary changes duringprocedure, than abrupt change in the medical device location on thedisplay.

In some embodiments, the patient may be instructed to stop beratingduring the capture of the images in order to prevent movements of thetarget area due to breathing. In other embodiments, methods forcompensating breathing movements during the capture of the images may beperformed. For example, the estimated poses of the fluoroscopic devicemay be corrected according to the movements of a fiducial marker placedin the target area. Such a fiducial may be a medical device, e.g., acatheter, placed in the target area. The movement of the catheter, forexample, may be determined based on the locating system. In someembodiments, a breathing pattern of the patient may be determinedaccording to the movements of a fiducial marker, such as a catheter,located in the target area. The movements may be determined via alocating system. Based on that pattern, only images of inhale or exhalemay be considered when determining the pose of the imaging device.

Reference is now made to FIG. 9B, which is a schematic illustration of atwo-dimensional (2D) grid structure of sphere markers 950, in accordancewith the method of FIG. 9A. 2D grid structure of sphere markers 950includes a plurality of sphere shaped markers, such as sphere markers960 a and 960 b, arranged in a two-dimensional grid pattern. Using a 2Dpattern, as opposed to a 3D pattern, may facilitate the pose estimationprocess. Furthermore, when for example, a patient is required to lie onthe markers structure in order to estimate the pose of the fluoroscopewhile scanning the patient, a 2D pattern would be more convenient forthe patient. In some embodiments, the shape of the markers may besymmetric and such that the projection of the markers on the image wouldbe the same at any pose the imaging device may be placed. Suchconfiguration may simplify and enhance the pose estimation processand/or make it more efficient. For example, when the imaging device isrotated around the markers structure, markers having a rotation symmetrymay be preferred, such as spheres. The size of the markers structureand/or the number of markers in the structure may be determinedaccording to the specific use of the disclosed systems and methods. Forexample, since the pose estimation is used to construct a 3D volume ofan area of interest within a patient, then the markers structure may beof a size similar or larger than the size of the area of interest. Insome embodiments, the pattern of a structure of markers according to thedisclosed methods and systems may be periodic, such as a grid, as shownin FIG. 2A. Using a periodic pattern structure of markers may furtherenhance and facilitate the pose estimation process and make it moreefficient. The structure of markers, as a fiducial, should be positionedin a stationary manner during the capturing of the fluoroscopic images.In an exemplary 2D grid structure of sphere markers, which may be usedin conjunction with the system of FIG. 1, the sphere markers diametermay be 2±0.2 mm and the distance between the spheres may be about15±0.15 mm isotropic.

The system of FIG. 1 may be configured to execute the method of FIG. 9A.The method of FIG. 9A, or a portion of it, may be in the form ofinstructions executed by a computing device, such as computing device125 of FIG. 1. The computing device may include one or more hardwareprocessors, one or more memories or storage devices and a display. Theone or more hardware processors may be configured to execute the stepsof this method. The one or more memories or storage devices may beconfigured to store these instruction and/or the fluoroscopic imagedata. The medical device and/or target may be displayed on the displayor on a separate monitor such as monitoring equipment 10 of FIG. 1.

From the foregoing and with reference to the various figure drawings,those skilled in the art will appreciate that certain modifications canalso be made to the present disclosure without departing from the scopeof the same. For example, although the systems and methods are describedas usable with an EMN system for navigation through a luminal networksuch as the lungs, the systems and methods described herein may beutilized with systems that utilize other navigation and treatmentdevices such as percutaneous devices. Additionally, although theabove-described system and method is described as used within apatient's luminal network, it is appreciated that the above-describedsystems and methods may be utilized in other target regions such as theliver, kidney or heart. Further, the above-described systems and methodsare also usable for transthoracic needle aspiration procedures.Additionally, although the above-described system and method isdescribed as used mainly with respect to soft-tissue targets, it isappreciated that the above-described systems and methods may be utilizedwith respect to non-soft-tissue targets, such as bones. Furthermore, theabove-described systems and methods are also usable with robotic surgerysystems, such as da Vinci® Surgical System currently sold by IntuitiveSurgical®. For example, and with reference to FIG. 1, system 100 may besuch a robotic surgery system. Operating table 20, on which the patientis positioned during surgery, may be incorporate in a cart including oneor more robotic arms. The one or more robotic arms may be coupled withone or more dedicated medical devices. Monitoring equipment 120 and/orcomputing device 125 may be incorporate in a controller, e.g., acomputerized console, allowing the surgeon to operate the one or morerobotic arms while viewing the inside of the patient. Detailedembodiments of the present disclosure are disclosed herein. However, thedisclosed embodiments are merely examples of the disclosure, which maybe embodied in various forms and aspects. Therefore, specific structuraland functional details disclosed herein are not to be interpreted aslimiting, but merely as a basis for the claims and as a representativebasis for teaching one skilled in the art to variously employ thepresent disclosure in virtually any appropriately detailed structure.

As can be appreciated a medical instrument such as a biopsy tool or anenergy device, such as a microwave ablation catheter, that ispositionable through one or more branched luminal networks of a patientto treat tissue may prove useful in the surgical arena and the presentdisclosure is directed to systems and methods that are usable with suchinstruments and tools. Access to luminal networks may be percutaneous orthrough natural orifice using navigation techniques. Additionally,navigation through a luminal network may be accomplished usingimage-guidance. These image-guidance systems may be separate orintegrated with the energy device or a separate access tool and mayinclude MRI, CT, fluoroscopy, ultrasound, electrical impedancetomography, optical, and/or device tracking systems. Methodologies forlocating the access tool include EM, IR, echolocation, optical, andothers. Tracking systems may be integrated to an imaging device, wheretracking is done in virtual space or fused with preoperative or liveimages. In some cases the treatment target may be directly accessed fromwithin the lumen, such as for the treatment of the endobronchial wallfor COPD, Asthma, lung cancer, etc. In other cases, the energy deviceand/or an additional access tool may be required to pierce the lumen andextend into other tissues to reach the target, such as for the treatmentof disease within the parenchyma. Final localization and confirmation ofenergy device or tool placement may be performed with imaging and/ornavigational guidance using a standard fluoroscopic imaging deviceincorporated with methods and systems described above.

While several embodiments of the disclosure have been shown in thedrawings, it is not intended that the disclosure be limited thereto, asit is intended that the disclosure be as broad in scope as the art willallow and that the specification be read likewise. Therefore, the abovedescription should not be construed as limiting, but merely asexemplifications of particular embodiments. Those skilled in the artwill envision other modifications within the scope and spirit of theclaims appended hereto.

What is claimed is:
 1. A system for constructing fluoroscopic-basedthree-dimensional volumetric data of a target area within a patient fromtwo-dimensional fluoroscopic images, comprising: a structure of markers;a fluoroscopic imaging device configured to acquire a sequence of imagesof the target area and of the structure of markers; and a computingdevice configured to: estimate a pose of the fluoroscopic imaging devicefor at least a plurality of images of the sequence of images based ondetection of a possible and most probable projection of the structure ofmarkers as a whole on each image of the plurality of images; andconstruct fluoroscopic-based three-dimensional volumetric data of thetarget area based on the estimated poses of the fluoroscopic imagingdevice.
 2. The system of claim 1, further configured to facilitateapproach of a medical device to the target area, wherein a medicaldevice is positioned in the target area prior to the acquiring of thesequence of images, and wherein the computing device is furtherconfigured to determine the offset between the medical device and thetarget based on the fluoroscopic-based three-dimensional volumetricdata.
 3. The system of claim 2, further comprising a locating systemindicating the location of the medical device within the patient,wherein the computing device comprises a display and configured to:display the target area and the location of the medical device withrespect to the target; facilitate navigation of the medical device tothe target area via the locating system and the display; and correct thedisplay of the location of the medical device with respect to the targetbased on the determined offset between the medical device and thetarget.
 4. The system of claim 3, wherein the computing device isfurther configured to: display a 3D rendering of the target area on thedisplay; and register the locating system to the 3D rendering, whereinthe correcting of the location of the medical device with respect to thetarget comprises updating the registration between the locating systemand the 3D rendering.
 5. The system of claim 3, wherein the locatingsystem is an electromagnetic locating system.
 6. The system of claim 3,wherein the target area comprises at least a portion of the lungs andthe medical device is navigated to the target area through the airwaysluminal network.
 7. The system of claim 1, wherein structure of markersis of a periodic pattern.
 8. The system of claim 1, wherein structure ofmarkers is of a two-dimensional pattern.
 9. The system of claim 1,wherein the target area comprises at least a portion of the lungs. 10.The system of claim 1, wherein the target is a soft-tissue target.
 11. Amethod for constructing fluoroscopic-based three dimensional volumetricdata of a target area within a patient from two dimensional fluoroscopicimages, the method comprising using at least one hardware processor for:acquiring, via a fluoroscopic imaging device, a sequence of images ofthe target area and of a structure of markers, wherein the structure ofmarkers is positioned between the patient and the fluoroscopic imagingdevice; estimating a pose of the fluoroscopic imaging device for atleast a plurality of images of the sequence of images based on detectionof a possible and most probable projection of the structure of markersas a whole on each image of the plurality of images; and constructingfluoroscopic-based three-dimensional volumetric data of the target areabased on the estimated poses of the fluoroscopic imaging device.
 12. Themethod of claim 11, wherein a medical device is positioned in the targetarea prior to the acquiring of the sequence of images, and wherein themethod further comprises using said at least one hardware processor fordetermining the offset between the medical device and the target basedon the fluoroscopic-based three-dimensional volumetric data.
 13. Themethod of claim 12, further comprising using said at least one hardwareprocessor for: facilitating navigation of the medical device to thetarget area via a locating system indicating the location of the medicaldevice and via a display; and correcting a display of the location ofthe medical device with respect to the target based on the determinedoffset between the medical device and the target.
 14. The method ofclaim 13, further comprising using said at least one hardware processorfor: displaying a 3D rendering of the target area on the display; andregistering the locating system to the 3D rendering, wherein thecorrecting of the location of the medical device with respect to thetarget comprises updating the registration of the locating system to the3D rendering.
 15. The method of claim 14, further comprising using saidat least one hardware processor for generating the 3D rendering of thetarget area based on previously acquired CT volumetric data of thetarget area.
 16. The method of claim 11, wherein the locating system isan electromagnetic locating system.
 17. The method of claim 13, whereinthe target area comprises at least a portion of the lungs and whereinthe medical device is navigated to the target area through the airwaysluminal network.
 18. The method of claim 13, wherein the structure ofmarkers is of a periodic pattern.
 19. The method of claim 13, whereinthe structure of markers is of a two-dimensional pattern.
 20. The methodof claim 13, wherein the target area comprises at least a portion of thelungs.
 21. The method of claim 13, wherein the target is a soft-tissuetarget.